MODIFIED LOCAL TERNARY PATTERN WITH CONVOLUTIONAL NEURAL NETWORK FOR FACE EXPRESSION RECOGNITION

Facial expression recognition (FER) on images with illumination variation and noises is a challenging problem in the computer vision field. We solve this using deep learning approaches that have been successfully applied in various fields, especially in uncontrolled input conditions. We apply a sequence of processes including face detection, normalization, augmentation, and texture representation, to develop FER based on Convolutional Neural Network (CNN). The combination of TanTriggs normalization technique and Adaptive Gaussian Transformation Method is used to reduce light variation. The number of images is augmented using a geometric augmentation technique to prevent overfitting due to lack of training data. We propose a representation of Modified Local Ternary Pattern (Modified LTP) texture image that is more discriminating and less sensitive to noise by combining the upper and lower parts of the original LTP using the logical AND operation followed by average calculation. The Modified LTP texture images are then used to train a CNN-based classification model. Experiments on the KDEF dataset show that the proposed approach provides a promising result with an accuracy of 81.15%

Selective local binary pattern with convolutional neural network for facial expression recognition

Variation in images in terms of head pose and illumination is a challenge in facial expression recognition. This research presents a hybrid approach that combines the conventional and deep learning, to improve facial expression recognition performance and aims to solve the challenge. We propose a selective local binary pattern (SLBP) method to obtain a more stable image representation fed to the learning process in convolutional neural network (CNN). In the preprocessing stage, we use adaptive gamma transformation to reduce illumination variability. The proposed SLBP selects the discriminant features in facial images with head pose variation using the median-based standard deviation of local binary pattern images. We experimented on the Karolinska directed emotional faces (KDEF) dataset containing thousands of images with variations in head pose and illumination and Japanese female facial expression (JAFFE) dataset containing seven facial expressions of Japanese females’ frontal faces. The experiments show that the proposed method is superior compared to the other related approaches with an accuracy of 92.21% on KDEF dataset and 94.28% on JAFFE dataset

Representasi Fitur Menggunakan Multi-scale Block Modified Local Ternary Pattern pada Pengenalan Wajah Berbasis Video

Pengenalan wajah merupakan sistem identifikasi personal berdasarkan data biometrik wajah. Beberapa penelitian telah dieksplorasi untuk menghasilkan sistem pengenalan wajah yang robust terhadap kriteria tertentu. Eksplorasi tidak hanya menitikberatkan pada algoritma klasifikasi namun juga algoritma ekstraksi fitur. Penelitian sebelumnya tentang eksplorasi ekstraksi fitur, menggunakan perhitungan dari keseluruhan gambar untuk menghasilkan fitur global, atau menggunakan perhitungan statistik dari sub-region gambar untuk menghasilkan fitur lokal. Selain pendekatan tersebut, variasi metode dari keduanya juga dieksplorasi untuk meningkatkan kemampuan representasi fitur dan menyelesaikan tantangan dari metode sebelumnya. Dengan melihat karakteristik data gambar yang memiliki variasi pencahayaan dan mengandung Gaussian/Poisson/Quantization noise, maka sistem pengenalan wajah yang bekerja pada dunia nyata dengan kondisi lingkungan tidak terkontrol akan memiliki tantangan di dalamnya. Penelitian ini mengeksplorasi solusi atas masalah tersebut. Penelitian ini mengusulkan metode representasi fitur bernama Multiscale Block Modified Local Ternary Pattern (MBMLTP) pada pengenalan wajah berbasis video. Kontribusi penelitian yaitu representasi fitur baru yang merupakan kombinasi fitur lokal dari modifikasi threshold yang adaptif pada Local Ternary Pattern (LTP) dan fitur global dengan konsep multiscale block. Metode LTP, salah satu metode yang memiliki sifat yang tidak sensitif terhadap noise dengan threshold yang ditentukan secara manual. Sedangkan konsep multi-scale block memiliki kemampuan menangkap representasi gambar secara global, sehingga memperkaya representasi fitur struktur dalam skala besar. Pengenalan wajah ini terdiri dari empat tahapan, yakni: deteksi ROI wajah, preprocessing, ekstraksi fitur dengan metode usulan, tahap klasifikasi. Uji coba menggunakan data video CCTV yang telah disimpan dan berisi seseorang sedang memasuki ruangan. Hasil pengujian menunjukkan sensitivity pengenalan wajah berbasis video adalah 100% dan sensitivity berbasis frame 90.28%. Dari hasil tersebut, dapat disimpulkan bahwa metode yang diusulkan menghasilkan representasi fitur yang lebih tahan terhadap variasi pencahayaan, dan noise daripada metode yang menghasilkan fitur lokal lainnya. ================================================================================================ Face recognition is personal identification system based on facial biometric data. Several research has produced facial recognition systems that are robust against specific criterions. Research has not only emphasizing on classification algorithm but also feature extraction algorithm. Previous research exploring feature extraction are using either calculation from the whole image to produce global feature, or statistical calculation from image’s sub-region to produce local feature. Beside those approaches, some research explored on variation of those two methods to increase the feature representation capability and to solve the challenges from previous methods. By using image data characteristic that has lighting variation and contains Gaussian/Poisson/Quantization noise, therefore facial recognition system that works on the real world with uncontrolled lighting environment poses some challenges. This research is intended to solve those challenges. This research intends to propose a feature representation method called “Multi-Scale Block Modified Local Ternary Pattern (MBMLTP)” on video-based facial recognition. The contribution of this research are new feature representation method that combines local feature from adaptive threshold modification on Local Ternary Pattern (LTP) and global feature using multiscale block concept. Local Ternary Pattern is a method that are insensitive towards noise in video dataset but require threshold determination, while multi-scale block has the capability to capture image representation globally, and therefore enriches the structure feature representation in big scale. This facial recognition consists of four steps, namely: face ROI detection, preprocessing, feature extraction using proposed method, and classification. The testing in this research are using CCTV video that has been store and consists of someone coming into the room. The testing showed sensitivity based on video are 100% and sensitivity based on frame are 90.28%. From those result, it can be concluded that the proposed methods produce a feature representation that are more robust against lighting variations and noises compared to other methods that uses other local features only